Real-time Learning when Concepts Shift

We are interested in real-time learning problems where the underlying stochastic process, which generates the target concept, changes over time. We want our learner to detect when a change has occurred, thus realizing that the learned concept no longer fits the observed data. Our initial approach to this problem has been to analyze offline approaches to addressing concept shifts and to apply them to real-time problems. This work involves the application of the Minimum Description Length principle to detecting real-time concept shifts. Introduction If enough consistent data can be obtained, standard machine learning algorithms can be applied to most learning problems in an offline, batch learning approach [5, 6, 7]. Our interest is in an online, sequential learning process, where the probabilistic functions that are generating the attributes and classifications of the world are changing over time. In the classic supervised learning problem [5, 9], it is generally stated that some stochastic function F(x) is generating an attribute vector x, based on a fixed probability distribution. The attribute vector, x, represents the salient features of the world. In a batch learning mode, the learner is given preclassified training examples based on a conditional distribution function F(y|x), where the values that y can take on are the classification values that would be associated with an instantiation of the attribute vector x. This function is unknown to the learner. Consequently, it is the job of the learner to attempt to approximate this function, F(y|x), as best it can by observing the training values and applying a learning algorithm. The online, sequential learning process presents several added difficulties. For example, since the learner is receiving it’s training data sequentially; it will need to repeatedly apply the learning algorithm until it is satisfied that it has converged to a good model of the world. This means that it has to store all of the previously encountered training vectors in some usable form. Due to the enormity of the datasets, some summarization technique must be used that does not sacrifice valuable information [3]. This summarization technique must also be chosen so that it does not inappropriately bias the next learning iteration toward a previously learned model. Our ultimate goal is to address uncertainty in real-time distributed computing problems with agent-based solutions. At the core of these agent-based solutions would be a realtime learning component that can detect and address shifts in the underlying target concept. Our initial approach to this problem has been to analyze offline approaches that address concept shifts and to apply them to real-time problems. Our first effort involves the application of the Minimum Description Length principle to detecting concept shifts. Sliding Window A current method used to address concept drift is the sliding window [4, 10] approach (figure 1). The idea is for the learner to only consider the most recent n observed examples when learning the concept. There are a number of parameters the learner must consider, such as the size of the window and whether the window size is adaptive or stable. The size of the window is of importance in that a small window would compromise the confidence in the learned concept. Learning a concept on a small data sample can easily lead to overfitting the learned concept to small aberrations in the data or simply overlooking entire elements of the target concept. Conversely, large window sizes lead to resource issues, in real time systems and may also leave the learned concept overly vulnerable to drift in the target concept. If the window size is large enough so that the range of enclosed attribute vectors drifts significantly, then the learned concept will perform poorly in a predictive capacity. Lastly, it seems that the sliding window approach is much better suited to concept drift, rather than concept shift. Concept drift occurs when the target concept changes gradually, while concept shift occurs when the target concept changes instantly. Instantaneous change in the target concept would leave the learned concept open to error until the entire window was drawn from the new target concept. Problem Domain The goal of our research is to create highly adaptable, real-time distributed computer systems. The benefits of this work could be realized through improved network data transmission speeds, greater autonomous control and coordination in satellite constellations and robot applications, and greater reliability in wireless communication, just to name a few. Even under controlled conditions, distributed computing issues, such as maintaining global state and determining causality among a series of events, are fraught with difficulty. There are known algorithms for addressing these problems, however they are designed to work in environments where communication reliability is guaranteed and processors never fail. These algorithms quickly fail when a distributed computing application is introduced into a volatile environment, in which communication links may be unstable, the number of nodes may change, or adversaries may introduce errors into the communication channel. Our research addresses the need for autonomous adaptability in real-time environments by introducing specialized intelligent agent technology into the distributed computing arena. Using this technology, we are providing a method for coping with uncertainty in order to address real-time adaptability issues. Time Figure 1. Addressing real-time concept drift/shift with a sliding window approach. The learner only considers the most recent n attribute vectors. Current attribute